1. Field of the Invention
The present invention relates generally to image/video display and processing systems, and in particular, to a method, apparatus, and article of manufacture for dynamically determining and utilizing a particular image resolution for a display and using such dynamically determined resolution to perform audio/video processing.
2. Description of the Related Art
Audio and video files, in the form of a plurality of digitized frames, are often very large and consume considerable time and bandwidth to display and process. Further, various different display devices may be used to view such digitized frames. However, while a digitized frame may have a high resolution, the display device may not be capable of displaying the frame at such a resolution. Accordingly, a lower resolution image is often used. However, the creation of the lower resolution image is time consuming and the newly created image may not utilize the full capabilities of the display device (i.e., the image resolution may be lower than the capabilities of the display device). Further, the processing of such high resolution images is processor intensive and consumes excessive amounts of time. Accordingly, what is needed is the capability to maximize the capabilities of a display device and perform processing in a quick and efficient manner. These problems may be better understood by describing frame resolutions and prior art methodologies for displaying and processing such frames.
Image data may be created, processed, and stored at various resolutions. As the resolution of an image increases, the size of the data needed to display/render that image also increases. 2K data has become the common format for the scanning and use of film images and commonly consists of a scan of a 35 mm negative image, frame line to frame line, full width of the film in a 4×3 aspect ratio. However, larger (i.e., higher resolution) images may also be used (e.g., 4K or 16K). Nonetheless, with large resolution images, a single frame may be beyond the address space of virtual memory in a 32-bit architecture.
In addition, with high resolution images, a display device or media player executing on a computer may not have the capabilities to display the image. For example, the physical area of a display screen may not be capable of displaying an entire image having a high resolution (e.g., a 4K, 16K or 42K image). For example, to display the full resolution image while displaying the complete image, a higher resolution display device may be necessary. Alternatively, only part of the image at the higher resolution may be displayed. To accommodate the partial images, the full resolution image may be broken up into various tiles of the same (or different) size wherein each tile contains a portion of the full resolution image. When processing or displaying a partial image, the tile(s) that encompass the desired portion of the file are utilized. The use and storage of tiles expedites the processing and display of such high resolution images.
Thus, processing is being driven by the display. If the display is about to display a particular frame, it first determines which tiles of that frame would appear (fully or partially) in the display area and then it will ask the processing subsystem for those frames. Once requested (and only at that point in time), image processing is done and only for those requested tiles. Thus, processing is saved by limiting the amount (i.e., not processing tiles that will not be visible on the screen) and timing of the processing.
In other prior art methods, to display an entire image (using a media player) on a display device (that is not capable of displaying complete high resolution images), the user may be forced to zoom out the image such that fewer pixels are viewed. Accordingly, the full resolution and details of the image are not being viewed by the user. Nonetheless, while the user is viewing the image at a lower resolution, the media player may still be processing and manipulating the large resolution file. Accordingly, any manipulations performed on the image are performed on the high resolution image thereby consuming considerable time and bandwidth.
As described above, when zooming out an image, a different version of the image (e.g., lower resolution image) may be displayed. However, prior art methods may also use the lower resolution image to conduct processing. A lower resolution image that will undergo image processing operations in place of original full resolution material is referred to as a proxy image. Further, the ratio of the full resolution image size to the proxy image size is referred to as the proxy scale.
One further prior art problem arises when selecting the proxy scale. For example, should the proxy scale for an entire clip be selected at a level below, equal to, or more than the current capabilities of a display device to be utilized? The artist will typically choose a proxy scale based on the processing load on their machine; the goal being to be able to work with the material as interactively as possible. Once decisions are made interactively, then the full resolution material is processed off-line to yield the final result. However, not all creative decisions can be done on lower resolution proxies. Accordingly, the artist may work with proxies for some processing and with full resolution material for other processing.
The prior art fails to provide an effective choice for either a single frame or a clip. In a different field of art, three-dimensional (3D) renderers may select a texture map that allows the image to look satisfactory but as small as possible. Further, such 3D rendering systems need to sample texture images that involves computing the average color of a given region of the texture image. Some 3D sampling algorithms may build a pyramid of proxies for a given texture image so that during rendering, computing the average color of large regions of the texture image can be done more efficiently (since each pixel of a proxy image is already an average of a larger group of pixels in the original). Based on the screen pixel being rendered, the 3D renderer is capable of predicting which texture proxy level to sample from so as to maximize rendering speed. However, such prior art 3D renderers are not utilized in 2D image processing systems and are specifically directed towards texture maps and average color calculations.